Smart Network Caches: Localized Content and Application Negotiated Recovery Mechanisms for Multicast Media Distribution

In the past fifteen to twenty years, two digital revolutions have been quietly occurring. The first is an effect of Moore's law of computing power; microprocessor power has grown to the point where inexpensive home computers can encode, decode, and display video. This new found ability to cheaply process video heralds the impending realization of digital convergence at the source and consumer, the end points of the distribution chain. Between the source and consumer lies the distribution channel. The second revolution-realizing digital convergence in the distribution channel-is only just beginning since the only widespread communication paradigm is that of point-to-point. Pointto-multipoint and multipoint-to-multipoint paradigms are proving to be much more difficult to deploy with the same flexibility, timeliness, and reliability that is associated with the point-to-point paradigm. In this dissertation, a new model called content and application negotiated delivery (CANDY) is developed to realize the point-to-multipoint and multipoint-to-multipoint communication paradigms. Two ideas are combined in this model: (1) every member of the session can be a cache or a server for every other member, and (2) the characteristics of the data and needs of the receiving application are used to assist in the delivery of data between session members. These two ideas have been explored individually at both the packet and stream levels, but they have not been explored simultaneously as the basis for a distribution system. By developing these ideas together, several new pieces of technology have been invented that advance the state of the art in terms of realizing digital convergence in the communications channel. A new receiver-driven multicast transport protocol is developed for the delivery of individual single streams of packets. This new protocol scales significantly better than existing ones. The protocol is further extended, applying the lessons at the packet level to the stream level, to develop a scalable set of methods that increase the ability for on demand access through "late joins" or catch ups. Finally, a number of integrated caching tools are developed to exploit, to full effect, the power afforded by the new transport protocol. Thesis Supervisor: Andrew B. Lippman Title: Associate Director, MIT Media Laboratory This work was supported by the members of the Singapore Program in Digital Media and Digital Life Consortium, AT&T, and Motorola.